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Students in introductory physics struggle with vector algebra and these challenges are often as¬ 
sociated with contextual and representational features of the problems. Performance on problems 
about cross product direction is particularly poor and some research suggests that this may be 
primarily due to misapplied right-hand rules. However, few studies have had the resolution to ex¬ 
plore student use of right-hand rules in detail. This study reviews literature in several disciplines, 
including spatial cognition, to identify ten contextual and representational problem features that 
are most likely to influence performance on problems requiring a right-hand rule. Two quantitative 
measures of performance (correctness and response time) and two qualitative measures (methods 
used and type of errors made) were used to explore the impact of these problem features on student 
performance. Quantitative results are consistent with expectations from the literature, but reveal 
that some features (such as the type of reasoning required and the physical awkwardness of using a 
right-hand rule) have a greater impact than others (such as whether the vectors are placed together 
or separate). Additional insight is gained by the qualitative analysis, including identifying sources 
of difficulty not previously discussed in the literature and revealing that the use of supplemental 
methods, such as physically rotating the paper, can mitigate errors associated with certain features. 

PACS numbers: 01.40.Fk, 41.20.Gz 


I. INTRODUCTION 

The use of physical mnemonics {e.g. right-hand rules) 
to remember the direction of various physical quantities 
has a history dating from the late 1800’s [T]. In the 
present day, most introductory physics sequences incor¬ 
porate right-hand rules (RHRs) to some extent, typically 
in rotational dynamics and/or magnetism. 

The goal of these various RHRs is arguably to un¬ 
load some of the mental processes required to deal with 
the 3-dimensional quantities related through vector cross 
products [5]. Yet, there is some evidence, and rampant 
speculation, that RHRs bring their own set of challenges 
that may be associated with particular representational 
or contextual features of the problem |3H5]. 

The current paper (based on the author’s disserta¬ 
tion i), explores the following question: 

How do contextual and representational prob¬ 
lem features impact student performance on 
problems requiring right-hand rules? 

The focus on contextual and representational depen¬ 
dence relies on two theoretical frameworks: resources and 
cognitive load theory. The resources framework claims 
that knowledge is composed of networks of small re¬ 
sources that can be activated in various ways depend¬ 
ing on the context [7] . Cognitive load theory claims that 
tasks with the same inherent difficulty (intrinsic cognitive 
load) may require more or less mental effort depending on 
the contextual and representational features of the task 


(extraneous and/or germane cognitive load) [8]. In the 
light of these theories, helping students use RHRs effec¬ 
tively requires a better understanding of how their use of 
RHRs varies with context and representation. 

In order to emphasize the use of RHRs, this paper fo¬ 
cuses on those RHRs associated with the generation of 
magnetic fields from moving charges and magnetic force 
acting on moving charges. At this point in the introduc¬ 
tory physics sequence, most students already have some 
familiarity with RHRs from rotational dynamics. 

Section|ll]uses literature from several research domains 
to identify the features that are most likely to affect stu¬ 
dent performance and what we would expect that impact 
to be. Section |Hl] discusses the design of this study, fo¬ 
cusing on operationalizing the features identified in Sec¬ 
tion |TT] and identifying appropriate measures of perfor¬ 
mance. Sections |IV| and [V] present quantitative and qual¬ 
itative analyses of the data, respectively. Finally, Sec¬ 
tion |VI| reviews the findings from both analyses and dis¬ 
cusses the instruction implications of this research. 


II. PREVIOUS RESEARCH RELEVANT TO 
THE USE OF RIGHT-HAND RULES 


There are several domains of research that have im¬ 
plications for understanding student use of RHRs. This 
study draws primarily on the following areas: 


conceptual understanding of magnetism (H AI, 


understanding of vector algebra (H B) , and 


• spatial cognition, specifically mental rotations and 
form perception (H C). 
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FIG. 1. Overview of the problem features identified from the literature and used in this study as predictors of performance. 
Individual features are in light blue (single quotations in text), with the variations beneath in white (italics in text). The 
features are grouped into larger categories (dark blue)) to facilitate discussion in the text. See Table |I] for the hypothesized 
impact of each feature on performance. 


Based on a synthesis of the research in these fields, this 
section identifies the contextual and representational fea¬ 
tures that are most likely to affect student performance 
on problems requiring the use of a RHR (Fig.[^ and what 
we expect that impact to be (Table |l|. In the text, the 
problems features will be identified with single quotation 
marks and the variations with italics. 


A. Magnetism 


There have been several attempts in recent years to de¬ 
velop and validate a conceptual survey of electromagnetic 
concepts analogous to the Force Concept Inventory (FCI) 
(e.g.. Ding et al. [S], Maloney et al. nni). Through these 
tools, as well as more qualitative methods, the literature 
on student understanding of electromagnetism identifies 
several key areas where students struggle with the fun¬ 
damental concepts of magnetism. 

One of the major sources of confusion is differentiat¬ 
ing between electric and magnet ic concepts [U fTOHI^ . 
According to Scaife and Heckler mm, this confusion 
was more common on magnetic force questions when the 
magnetic field was represented by magnetic poles than 
when it was represented with field lines (‘Field’). The 
magnetic poles representation was also related to a sign 
error resulting from reversing the direction of the mag¬ 
netic field due to those poles (south to north instead of 
north to south). They also identified two other sources 
of these non-random sign errors: “a confusion in choice 
and execution of the several right-hand rules available” 
and an unawareness of the non-commutative nature of 
the cross product. 

There is also evidence that students struggle more 
with electric field problem than with electric force prob¬ 


lems |13j . suggesting the possibility of a similar pattern 
with magnetic field and force problems (‘Context’). 

In addition to confusion between electric and magnetic 
field and forces, conventional wisdom also assumes that 
the need to reverse the direction of the cross product 
for a negative charge adds an additional layer of diffi¬ 
culty (‘Charge’). Thus, we would expect that partici¬ 
pants would perform better on problems with a positive 
charge or a current-carrying wire than on problems with 
a negative charge. 

Also, there are many different RHRs that one can use, 
particularly for different current distributions, such as 
a straight wire or a loop of wire (‘Shape’). One might 
expect the shape of the current distribution to affect the 
type of RHR used, while not necessarily affecting the 
accuracy of the response. 


B. Vector Algebra 

The research on student understanding of vector alge¬ 
bra has included studies on student conceptual under¬ 
standing of vector properties m and the design and 
assessment of diagnostic tools to assess the level of in¬ 
coming university students’ ability to do vector oper¬ 
ations dSHn]. More relevant to the current study 
are those that explore the sources of student difficulty 
with basic vector concepts and operations HBUS] and 
those that compare performance on vector operations 
in different contexts m MM, with different repre¬ 
sentations m [13H15], and/or with different interven¬ 
tions [2H1 [?7] . 

The majority of these studies focus on vector addi¬ 
tion and subtraction using an arrow representation in a 
mechanics contexts {e.g., force, displacement, momen- 
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TABLE I. Hypothesized impact of predictors on performance. 
All except the last three (which serve as control variables) are 
shown in light blue in Fig.[^ Angle, Charge, Shape, and Field 
were treated separately in the quantitative analysis due to the 
constraints of the data. 


Predictor 

Hypothesis 

Context 

Participants will perform better on non¬ 
physics questions than on magnetic held or 
magnetic force questions and may perform 
better on magnetic force than on magnetic 
held questions. 

Type 

Participants will perform better on questions 
that ask for the cross product result (forward 
reasoning) than on those that ask for one of 
the given vectors (backward reasoning). 

Tails 

Participants will perform better on questions 
with vectors presented tail-to-tail than on 
those presented head-to-tail. 

Separation 

Participants will perform better on questions 
where the vectors are together than on those 
separate in space. 

Comfort 

Participants will perform better on questions 
with physically easy orientations than on 
those with awkward orientations. 

Plane 

Participants will perform better on questions 
in the xy plane than on those in a non-xy 
plane {xz or yz plane). 

Angle 

Participants will perform better on questions 
with perpendicular vectors than on those 
with acute or obtuse vectors. 

Charge 

Participants will perform better on questions 
with positive charge or a current-carrying 
wire than on those with negative charge. 

Shape 

Participants will use a different right-hand 
rule for a straight current-carrying wire and 
loop of current, but there will be no difference 
in performance. 

Field 

Participants will perform better on questions 
where the magnetic field is represented by 
vectors than on those with magnets. 

Order 

The order of the questions will be positively 
correlated with performance due to practice 
effects. 

Experience 

Participants with more experience (2nd 
group) will perform better than participants 
with less experience (1st group). 

Spatial 

Spatial ability will be positively correlated 
with performance. 


turn, etc.). Zavala a nd colleagues [131 HZl HU HU HH] and 
Scaife and Heckl^ [H [12] are have also studied vector 


multiplication, to some extent, and within the context of 
electromagnetism. However, only Scaife and Heckl^ [ij 
had the resolution to actually observe the use of RHRs. 

There are clear indications from this research that 
performance on vector algebra problems is both 
representation-dependent and context-dependent. One 
common source of difficulty in graphical vector addition 
and subtraction is the need for parallel transport: mov¬ 
ing the vectors in space while preserving length and di¬ 
rection [HHaiMj. Many RHRs assume that the vectors 
are tail-to-tail. Thus, if the vectors are presented head- 
to-tail (‘Tails’) and/or separated in space (‘Separation’), 
one would expect students to have more difficulty than 
if they are presented tail-to-tail and placed together. 

In addition, students tend to ignore directional infor¬ 
mation and will often use an incorrect angle or trigono¬ 
metric function to find components or calculate the mag¬ 
nitude of vector products. While using an inappropriate 
trigonometric function may partially be due to confusion 
between dot products and cross products, Barniol and 
Zavala m found the use of the correct trigonometric 
function was much higher on dot product problems than 
on cross product problems. For cross products, ques¬ 
tions where the vectors are not at right angles (‘Angle’) 
are likely more challenging than those where the vectors 
are completely perpendicular to each other. 

Several of the studies mentioned here examine perfor¬ 
mance on vector algebra problems in one or more physics 
context and compare it to performance in a non-physics 
(or math) context. In some cases, it appears that the 
physical situation may cue appropriate resources for stu¬ 
dents [31]. However, when asked to find the direction 
from a cross product, there is evidence that students 
struggle more when the problem is in a physics context 
than when it is in a non-physics context [5] (‘Context’). 

Finally, there is a clear consensus in this literature that 
vector subtraction is more challenging than vector addi¬ 
tion for students (although the alignment of the vectors 
makes a difference) [35]. Particularly with the arrow rep¬ 
resentation, vector subtraction is sometimes taught as if 
one were reasoning backward from vector addition {e.g., 
for A — B, what vector should be added to B in order to 
get A?). Thus, we would expect that cross product prob¬ 
lems that require reasoning backward {e.g., given the re¬ 
sultant and one initial vector, what is a possible direction 
for the other initial vector?) would be more challenging 
than those that require forward reasoning (‘Type’). 


C. Spatial Cognition 

Since a vector cross product is inherently three- 
dimensional, dealing with cross products often requires 
spatial reasoning analogous to performing a mental rota¬ 
tion. The use of a physical RHR introduces a kinesthetic 
element to this reasoning, but it does not eliminate the 
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spatial nature of the cognition required. There is a large 
body of literature dealing with spatial cognition that pro¬ 
vides some indications of what factors might influence 
the use of a RHR. Due to the breadth of this field, this 
discussion is restricted to a few key studies that have a 
direct bearing on the use of RHRs. 

According to Klatzky and Wu [3], the primary cogni¬ 
tive issue in using a RHR is one of coordinating and align¬ 
ing reference frames. They identify several frames that 
must be aligned in order to appropriately use a RHR: 
those defined by the vectors, by the page, and by the 
hand relative to the body. From this perspective of coor¬ 
dinating reference frames, they also provide a task anal¬ 
ysis for determining the direction of the cross product of 
two vectors U x V: 


“The performer must (a) create a reference 
frame in which U and V lie on a common 
plane at an arbitrary location in self-defined 
space, (b) treat the hand as an external ob¬ 
ject within the vector-defined frame and map 
hand-centered coordinates into that frame so 
that the wrist lies at the vector intersection, 
the hand’s cross-section is aligned with U, 
and V is closer to the palm than the back 
of the hand; (c) observe the direction of the 
thumb in space and align it with the third di¬ 
mension in the vector-defined coordinate sys¬ 
tem.” (p. 171) [3] (Note: this RHR is referred 
to as RHR: Type:Standard in Table [V|) 


[Klatzky and Wu| state that the primary difficulties of 
this task are creating the frame from the vectors and 
getting one’s hand into the appropriate position in that 
frame. While the use of the body may often facilitate 
cognition [53], Klatzky and Wu argue that the spatial 


reasoning required for the more intricate postures is cor¬ 
respondingly complex and these unusual positions could 
increase the cognitive difficulty instead of decrease it. 

Other teachers and researchers have also suggested 
that difficulties with implementing a right hand rule 
are mainly kinesthetic: the accidental use of one’s left 
hand [30]; losing track by becoming so focused on the 
manipulation of the hand ED; the physical difficulty of 
aligning one’s body in the proper orientation [5|; and in 
some cases, the inability to use one’s hand at all [55]. All 
of these authors assume that physical awkwardness con¬ 
tributes to difficulty with RHRs, although none provide 
experimental evidence to support this assumption. 

Research on alignment has also identified a human sen¬ 
sitivity to orientation, usually in the contexts of form 
perception [33H35j : imagined rotation, translation, and 
transformation [361 [37]; and the kinesthetic aspects of 
spatial reasoning [35] [33] . This literature indicates that 
the “top” of an image, as well as the vertical direction, 
have a special place in human perception and in the 
ability to perform mental rotations. For example, jPanij 
and colleagues [33H35] found that a vertical axis of ro¬ 
tation — and to a lesser extent, the horizontal — are 


the only directions where participants could consistently 
perform mental rotations accurately. Shelton and Mc¬ 
Namara m also found that salient environmental prop¬ 
erties (such as walls and furniture) strongly influenced 
mental representation of the space and performance on 
spatial tasks. These results, along with the task analysis 
discussed above [3| , suggest that the use of a RHR would 
be more difficult in situations when the vectors are not 
aligned with the local reference frame (e.g., vectors are 
at an angle and therefore not aligned with the axes of the 
paper) or when the axis of rotation is not vertical (e.g., 
one of the given vectors is not in xy-plane). 

Despite these implications from spatial cognition re¬ 
search, Scaife and Heckler |4] did not see any measurable 
impact due to vector orientation while conducting indi¬ 
vidual interviews with students solving magnetic force 
problems. In order to resolve this difference in the liter¬ 
ature, the current study attempts to separately address 
three different aspects of orientation: 

• physical awkwardness (‘Comfort’); 

• axis of rotation or plane of the vectors (‘Plane’); 

• alignment with the local reference frame (‘Angle’). 

The results show that by treating these issues separately, 
each aspect of orientation does have a measurable effect 
on performance that is consistent with the spatial cogni¬ 
tion literature. 


D. Summary of previous research 

Drawing on three different research domains, this sec¬ 
tion has identified ten different problem features that are 
likely to impact performance. These features and their 
hypothesized impact on performance are summarized in 
Fig. [2 and Table|l| respectively. In order to facilitate dis¬ 
cussion in the text, the features have been grouped into 
slightly larger categories, as shown in Fig. 

• Kind of Question —broad characteristics of the 
problem: the physics (or non-physics) context 
(‘Context’) and the type of reasoning (‘Type’). 

• Orientation —features related to spatial cognition: 
physical awkwardness (‘Comfort’), the plane of the 
given vectors (‘Plane’), and the angle between the 
given vectors (‘Angle’). 

• Parallel Transport —features indicating whether a 
vector will need to be moved: whether the vectors 
are head-to-tail or tail-to-tail (‘Tails’), and whether 
they are together or separate (‘Separation’). 

• Physics —features most closely related to the physi¬ 
cal situation: the sign of the charge (‘Charge’), how 
the magnetic field is represented (‘Field’), and the 
shape of the current distribution (‘Shape’). 

Also included in Table |T] are three control variables that 
will be discussed in the following section. 
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III. STUDY DESIGN 

In order to assess the impact of the features identi¬ 
fied in Section |ll] on student performance, we need to 
first operationalize how we will distinguish between the 
variations within a given feature, as well as how we will 
measure performance. After providing details about the 
study context, including the participants and tasks, this 
section will discuss how the problem features and perfor¬ 
mance were operationalized in this study. 

A. Study context 

The author conducted individual semi-structured in¬ 
terviews with participants recruited from three sec¬ 
tions [23] of a calculus-based introductory physics se¬ 
quence, all using the third edition of Matter and Interac¬ 
tions [44] , Fifteen students were interviewed within two 
weeks of finishing the unit on magnetic fields. After fin¬ 
ishing the unit on magnetic force, an additional twelve 
students were interviewed. The two groups were distin¬ 
guished from each other to control for any effect due to 
more exposure to the content (‘Experience’). 

Every interview was recorded from two different per¬ 
spectives (overhead and side view) and there were two 
major tasks given to participants. The first task was the 
Cube Comparison Test |45| , which was used to provide a 
measure of individual spatial ability that could be con¬ 
trolled for in the quantitative analysis (‘Spatial’). The 
second task was a think-aloud protocol where partici¬ 
pants solved many cross product direction questions and 
the features in Fig. [l] were varied across relevant prob¬ 
lems. Given the large number of problems, there was the 
possibility of improvement over the course of the inter¬ 
view (a practice effect). To address this possibility, each 
participant received the problems in the same, partially 
randomized order |46j and the quantitative analyses con¬ 
trolled for this order (‘Order’). 

At the end of the interview, the interviewer asked every 
participant four follow-up questions: 

1. Describe the right-hand rule(s) that you use. 

2. How does the direction of A x i? compare to the 
direction of B x A? 

3. On a scale of one to ten, what is your comfort level 
or confidence with using the right-hand rule? 

4. Is there anything you found difficult about these 
problems that you have not already mentioned? 

Although not analyzed in detail, responses to these ques¬ 
tions were used to supplement the qualitative analysis. 
For example, responses to Question 1 were used as a 
starting point for descriptions of different types of right- 
hand rule and Question 2 provided an explicit probe as¬ 
sessing whether the participant was aware of the non- 
commutative nature of the cross product. 
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FIG. 2. Examples of Non-physics (row 1), Magnetic Field 
(row 2), and Magnetic Force (row 3) questions where the 
problem involves working Forward (column 1) and working 
Backward (columns 2 and 3). Problem statements for rows 2 
and 3 have been abbreviated for space. 


B. Operationalizing problem featnres 

This section will provide examples of how the problem 
features in Fig. [^were varied in this study. 

1. Kind of question: ‘Context’ and ‘Type’ 

Questions were given in one of three contexts. 

• Magnetic Field questions involved the generation 
of magnetic field from moving charges. 

• Magnetic Force problems involved the force on 
moving charges due to an external magnetic field. 

• Non-physics problems were framed as C = A x B, 
with no explicit physics context. 

In each context, there were questions that involved 
Forward reasoning and questions that involved Back¬ 
ward reasoning. Reasoning forward meant determining 
the cross product resultant vector from the given vectors 
(“What is the direction of A x B?”). Reasoning backward 
questions provided the resultant vector (C) and asked for 
one of the initial vectors (either A or B). In physics con¬ 
texts, a Backward question might also ask for the sign of 
the charge or direction of current. Figure]^ shows exam¬ 
ples of Forward and Backward problems for all contexts. 
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FIG. 3. The same orientation, (+y) x (—x) = +z, with the 
vectors Head-to-tail, Tail-to-tail, Together, and Separate. 


For logistical reasons, problems were presented to stu¬ 
dents in several sets based on ‘Context’ and ‘Type.’ Par¬ 
ticipants first answered the Magnetic Field problems, 
with Forward and Baekward questions intermixed. The 
second group of interviewees then answered Magnetic 
Force questions, again with Forward and Backward ques¬ 
tions intermixed [47) . After the physics problems were 
completed, both groups answered the Non-physics ques¬ 
tions, which were given as two sets of 25 Forward ques¬ 
tions and two sets of 10 Backward questions. 


2. Parallel Transport: ‘Tails’ and ‘Separation’ 

Problems were also varied in terms of the position of 
the vectors in space. This included whether the vectors 
were presented as Tail-to-tail or Head-to-tail, as well as 
whether they were Together or Separate in space. Fig¬ 
ure 1^ shows examples of the various possible combina¬ 
tions of these two features for one orientation. The 
non-physics problems included all of these combinations, 
whereas the physics problems included only those that 
were consistent with how physics problems are typically 
presented. Also, for Magnetie Field problems, the posi¬ 
tion vector was not provided explicitly, only the observa¬ 
tion location was indicated (see Fig. [|d for an example). 


3. Orientation: ‘Comfort’, ‘Plane’, and ‘Angle’ 


As discussed in Section |IIC| there are three aspects 
of the orientation of the vectors that were likely to con¬ 
tribute to difficulty: physical awkwardness (‘Comfort’), 
the plane of the vectors (‘Plane’), and the angle between 
the vectors (‘Angle’). There are 24 different orientations 
of perpendicular, on-axis vectors in a 3-dimensional or¬ 
thonormal coordinate space (Fig.|^. In order to restrict 
the possibilities and to quantify the level of physical awk¬ 
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FIG. 4. All possible cross product combinations for on-axis, 
perpendicular vectors {A x B). The green orientations were 
the five orientations with the lowest average ranking for physi¬ 
cal awkwardness {Easy orientations) and the blue orientations 
were those with the highest average ranking for physical awk¬ 
wardness {Awkward orientations). 


wardness for various orientations, three physics graduate 
students rated their physical discomfort when using a 
right-hand rule to find the cross product direction for 
each of these 24 orientations. They rated them on a 
5-point scale, with 1 being physically easy and 5 being 
physically awkward. The rankings for each orientation 
were averaged and the five orientations with the high¬ 
est rank {Awkward orientations) and the five orientations 
with the lowest rank {Easy orientations) were used as a 
basis set for all of the interview problems. 

It is important to note that whether an orientation 
is physically easy or physically awkward is dependent 
on both the method used and the errors made. Dif¬ 
ferent RHRs require different hand positions and there 
are additional complications for physics questions, such 
as whether one flips for an electron at the beginning or 
end of the problem. There are also numerous errors that 
change the awkwardness of the problem, such as reversing 
the order of the cross product {B x A instead oi Ax B). 
However, using these ten orientations allowed for a first- 
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FIG. 5. Examples of physically awkward orientations in the 
xy and non-xy planes. 
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FIG. 6. Examples of one orientation, (+y) x {—x) = +£, with 
the vectors presented as perpendicular and non-perpendicular 
(with both acute and obtuse angles). 


order comparison of the differences in performance on 
physically easy and physically awkward problems. 

Among the ten orientations identified as Easy or Awk¬ 
ward, there were physically easy orientations in the xy 
and yz planes and physically awkward orientations in the 
xy, yz, and xz planes. Thus, combining the yz and xz 
orientations into “non-xy” orientations allows us to sep¬ 
arately consider of the relative impact of the plane of the 
vectors and physical discomfort on performance. As an 
example, Fig. shows physically awkward orientations 
in the xy and non-xy planes. 

In order to limit the interview to one hour, the an¬ 
gle between the vectors was only varied for the forward 
reasoning questions in a non-physics context. Also, to 
keep orientations similar to those that students might 
see in typical physics problems, only orientations in 
xy plane were varied with respect to angle. There was 
a mix of acute and obtuse angles (combined into a non¬ 
perpendicular category) that were compared to the ma¬ 
jority of problems which had perpendicular orientations. 
Figure shows the same basic orientation with perpen¬ 
dicular, acute, and obtuse angles. 


4- Physics features: ‘Charge’, ‘Shape’, and ‘Field’ 



(c) 


FIG. 7. Examples of the variation in physics features. The 
‘Sign’ of the charge, which was varied between the positive 
and negative charges in (a) and the current-carrying wires in 
(b). The ‘Shape’ of the current-carrying wire in (b) were also 
varied between straight and circular. Finally, the represen¬ 
tation of magnetic ‘Field’ was varied between a vector/arrow 
representation and a bar magnet representation, shown in (c). 


point charge {Positive), a negative point charge {Neg¬ 
ative), and a current-carrying wire {Current) with the 
direction of conventional current indicated. The shape 
of the current-carrying wire was only varied for the mag¬ 
netic field problems and included straight wires {Straight) 
and both square and circular current loops {Loop). The 
representation of the magnetic field was only varied for 
the magnetic force problems and included either Vector 
or Magnet, analogous to the field lines and magnetic poles 
used in Scaife and Heckler H]. Figure shows examples 
of each of these features. 


There were three main physics features: the sign of the Measures of performance 

charge (‘Charge’), the shape of the current-carrying wire 

(‘Shape’), and the representation of the magnetic field In order to address the question of the how each feature 
(‘Field’). The variations in charge included a positive discussed above will impact performance, one must first 


































have a way to measure performance. For many problems, 
the most coarse measure of performance is whether the 
question was answered correctly. Yet, even correctness is 
not as straightforward as one might expect. For example, 
how does one distinguish between answering a question 
correctly on the first attempt and answering correctly on 
a final attempt after taking multiple different approaches, 
some of which may be inappropriate? In addition, cor¬ 
rectness may not be the most illuminating measure of 
performance. The current study used four different mea¬ 
sures of performance in order to provide a more holistic 
perspective: correctness of the final response, response 
time, methods used, and types of errors made. 

The first two of these measures — the correctness of 
the final response and the time taken to give that re¬ 
sponse — provide quantitative data about performance. 
Correctness (or accuracy) and response time (or latency) 
are often used in the spatial cognition literature as cor¬ 
related measures of cognitive difficulty. For example, it 
is assumed that the process of aligning reference frames 
comes at a measurable cost: the more mis-aligned the 
reference frames, the less accurate the response and the 
longer the response time will be [48]. 

Choosing the final response allowed for the possibility 
of participants changing their response throughout the 
problem and made correctness a binary variable. Prob¬ 
lems where a participant did not provide a final response 
{N = 33) were not included in the analysis. 

Response time was measured in seconds using video 
time stamps. The ‘Start’ was defined as the moment 
the participant first saw the question. This was done to 
address the fact that some participants would read the 
entire problem statement before trying to solve it, while 
others would jump into the problem and refer back to 
the problem statement only as needed. This definition of 
start time likely inflates the response time for the physics 
questions compared to the non-physics questions due to 
the amount of additional reading. The operational defi¬ 
nition for the ‘End’ was the first instance of the partici¬ 
pant’s final response. 

Analysis of this data involved regression analyses for 
correctness and response time to test the hypothesized 
impact of the problem features discussed in Table The 
details of these analyses and their results are discussed 
in Section nvi 

The other two measures—the methods participants 
used to solve the problem and the types of errors that 
participants made while solving the problem—were iden¬ 
tified from the data using an iterative coding process. 
This process and the codes that emerged are discussed 
in Section [vj (with more detailed code descriptions in the 
appendix). These qualitative measures provides some of 
the nuance that is lost with the coarse-grained measures 
of correctness and response time. 


IV. QUANTITATIVE ANALYSES 
A. Overview of quantitative analyses 


The impact of the problem features on the quantita¬ 
tive measures of performance—correctness and response 
time—was assessed using regression analyses. As defined 
in Section III C[ correctness is binary and thus logistic re¬ 
gression was appropriate. In contrast, response time is a 
continuous variable with a Poisson-like distribution, so a 
standard least squares regression analysis was used with 
the logarithm of response time as the dependent variable 
(log-linear regression). 

Since the focus of this study is the use of RHRs, prob¬ 
lems where a participant did not use a RHR were elimi¬ 
nated from the analyses (these problems were identified 
using the qualitative coding discussed in Section]^. Af¬ 
ter these problems had been eliminated, N = 2119 prob¬ 
lems remained for use in the analyses. 

Each regression included ten predictors: the first six 
problem features in Table the three control variables at 
the bottom of Table |l] and the other quantitative mea¬ 
sure of performance. As mentioned previously, it is a 
standard assumption in spatial cognition research that 
accuracy and latency are correlated measures of cogni¬ 
tive difficulty. To validate this assumption for the current 
study, correctness and response time were used as predic¬ 
tors for each other in the respective regression analyses. 
The results from the control variables will be presented 
as a part of the regression models for completeness, but 
the discussion here will focus only on the six problem 
features [15] . 

The remaining four problem features from Table [l| 
were not included in the regression analyses because they 
were not varied on all problems. The physics features 
(‘Charge’, ‘Field’, and ‘Shape’) could not be varied the 
same way across all contexts and in order to keep the 
interview to a reasonable length, ‘Angle’ was only varied 
for non-physics working forward problems. To assess the 
impact of these four features, the author performed an 
analysis of variance (ANOVA) F test for response time 
and a Likelihood Ratio test for correctness for the rel¬ 
evant subset of problems where each feature was varied. 

It is important to note that the data are such that a 
strict interpretation of the results is impossible {e.g., not 
all features were varied on all problems) and that these 
regression analyses are designed to provide a first-order 
assessment of how individual problem features impact 
performance on problems requiring a RHR. 


B. Results of quantitative analyses 

The logistic regression for correctness is presented in 
Table [nl including model level statistics and odds ratios. 
For logistic regression, B? or U is the proportion of the 
uncertainty for the Whole Model Fit. For this model, 
there is only 12% uncertainty attributed to the fit. Odds 
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TABLE II. Logistic regression for correctness. The odds are 
for correct versus incorrect, where odds > 1 indicate a posi¬ 
tive impact on correctness and odds < 1 indicate a negative 
impact on correctness. For continuous terms, the unit odds 
are per unit change in contrast and the range odds are per 
change in contrast over entire range. For categorical terms, 
odds are for Level 1/LeveI 2, i.e., for odds > 1, the Level 1 
category is more likely to be correct than the Level 2 category 

Model level statistics 


(U) 0.1254 

Observations (N) 2119 

statistic p value < 0.0001 


Continuous terms 


Term 

Unit Odds 

Range Odds 

p value 

Log (T) 

0.53 

0.042 

< 0.0001 

Spatial 

1.0 

4.6 

< 0.0001 

Order 

1.0 

1.3 

0.46 


Categorical terms 

Term 

Level 1 

Level 2 

Odds 

p value 

Experience 

1st group 

2nd group 

1.2 

0.12 

Context 

Physics 

Non-physics 

0.96 

0.86 

Type 

Forward 

Backward 

1.6 

0.0047 

Comfort 

Easy 

Awkward 

1.5 

0.0070 

Plane 

xy 

Non-xy 

1.5 

0.0046 

Tails 

Tail 

Head 

1.0 

0.84 

Separation 

Together 

Separate 

1.1 

0.72 


ratios are included since parameter estimates are not as 
easily interpreted for logistic regression. Odds greater 
than one indicate a positive impact on correctness and 
odds less than one indicate a negative impact on correct¬ 
ness. For continuous terms, unit odds represent the im¬ 
pact on correctness per unit change in the term, while the 
range odds indicate the effect on correctness per change 
in the term over the entire range. For categorical terms, 
the odds specify the probability of getting a correct re¬ 
sponse with the first level divided by the probability of 
getting a correct response with the second level (i.e., for 
odds > 1, the Level 1 category is more likely to be correct 
than the Level 2 category). 


The odds for all contrasts were in a direction consis¬ 
tent with the hypotheses in Table |Tj with the exception of 
‘Experience’ and this may be due to interference effects 
between concepts of magnetic field and magnetic force 
(discussed in the qualitative analysis in Section VC). 
However, the impact of most of these predictors on cor¬ 
rectness was not as significant as their impact on response 
time (see Table III). The problem features that appeared 
to have the largest impact on correctness were the type of 
reasoning required by the problem, the physical discom¬ 
fort of the orientation and the plane of the given vectors. 


TABLE III. Least squares regression for Log (T). A positive 
estimate indicates that the category in square brackets con¬ 
tributed to a longer response time than the other category 
associated with that term. For example, the positive esti¬ 
mate for Type[Backward] implies that Backward problems 
took more time than Forward problems, whereas the negative 
estimate for Correctness [Correct] implies that Correct prob¬ 
lems took less time than Incorrect problems. 


Model level statistics 

(adjusted) 

0.7080 


Observations (N) 

2119 


F statistic p value 

0.0000 


Term 

Estimate 

p value 

Intercept 

-f3.5 

0.0000 

Correctness [Correct] 

-0.087 

< 0.0001 

Spatial 

-0.013 

< 0.0001 

Order 

-0.013 

< 0.0001 

Experience [1st group] 

-fO.lO 

< 0.0001 

Context [Magnetic field] 

-f0.50 

< 0.0001 

Context [Magnetic force] 

-fO.48 

< 0.0001 

Type [Backward] 

-f0.18 

< 0.0001 

Comfort [Awkward] 

-f0.032 

0.0060 

Plane [Non-xy] 

-f0.094 

< 0.0001 

Tails [Head] 

-f0.012 

0.4246 

Separation [Separate] 

-f0.032 

0.0381 


The log-linear regression for response time is presented 
in Table 11111 which includes model level statistics and 
parameter estimates. This model accounted for approxi¬ 
mately 70% of the variance in response time. Although it 
is difficult to strictly interpret significance due to the lim¬ 
itations of the data, the results show that all of the main 
effects impacted response time in a direction consistent 
with the hypotheses in Table |T] and all were significant at 
the p < 0.01 level except for Tails and Separation, the 
two features associated with parallel transport. 

As discussed in Section IV A[ there were several fea¬ 
tures which were not varied across all problems (‘Angle’, 
‘Charge’, ‘Shape’, and ‘Field’) and thus, not included 
in the regression analysis. To assess the impact of each 
of these features, the author performed an analysis of 
variance (ANOVA) F test for response time and a Likeli¬ 
hood Ratio test for correctness for the relevant subset 
of problems where that feature was varied. The relevant 
subset for each of these features was: 


• ‘Angle’: all working forward, non-physics problems 
in the a; 2 /-plane {N = 723); 

• ‘Charge’: all magnetic field and magnetic force 
problems {N = 533); 

• ‘Field’: magnetic force problems where the mag¬ 
netic field was represented {N = 105); and 

• ‘Shape’: magnetic field and magnetic force prob¬ 
lems with a current-carrying wire {N = 160). 
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TABLE IV. Results for the analysis of variance (ANOVA) F 
test for response time and Likelihood Ratio test for correct¬ 
ness for four features (Angle, Charge, Shape, and Field). The 
N valne reflects the number of relevant problems for which 
each feature was varied. 


Term 

N 

F test 
p value 

test 
p value 

Angle 

723 

0.14 

0.041 

Charge 

533 

0.071 

0.035 

Field 

105 

0.34 

0.0025 

Shape 

160 

0.27 

0.33 


The p-values for each of these analyses are shown in 
Table |IV[ As expected, for different shapes of current- 
carrying wires, there was no significant difference in cor¬ 
rectness {p = 0.33) or response time {p = 0.27). For the 
other features, results were in a direction consistent with 
the hypotheses in Tabled but differences in correctness 
(x^) were significant at p < 0.05 level whereas differences 
in response time (F) were not. 


C. Discussion of quantitative analyses 


very low (< 30%). Thus, the lack of significance seen 
here may be due to an overall improvement in under¬ 
standing about cross products from the first semester to 
the second semester of the introductory sequence. 

Alternatively, the lack of significance may be due to 
the difference in physics context (torque vs. magnetism). 
Van Deventer found that in the torque context, the 
most common incorrect answer was that the direction 
was counterclockwise (consistent with the direction the 
pulley would move due to the torque applied). Since stu¬ 
dents have much more everyday experience with torque 
concepts than they do with magnetism concepts, it may 
be that the torque context cues inappropriate resources 
that the magnetism and non-physics contexts do not. 

2. Parallel Transport: Tails and Separation 

One of the more surprising results of this quantita¬ 
tive analysis is that the features associated with paral¬ 
lel transport appeared to have the least impact on both 
correctness and response time. Despite literature show¬ 
ing that students struggle when vector operations require 
moving the vectors {e.g., Hawkins et al. |23j). it appears 
that parallel transport may not be as much of an issue for 
cross products as it is for vector addition and subtraction. 


The results from the quantitative analyses were gener¬ 
ally consistent with expectations based on the literature 
and conventional wisdom. This section will briefly ad¬ 
dress the few exceptions as well as highlight points of 
interest that warrant particular notice in the context of 
the questions guiding this study. 


1. Kind of question and physics features 


The first thing of note is that the type of reasoning 
required (‘Type’) had one of the most significant im¬ 
pacts on both response time {p < 0.0001) and correct¬ 
ness {p = 0.0047). As expected, problems that require 
reasoning backward are significantly more difficult than 
those that require forward reasoning. Given its signifi¬ 
cance, the qualitative discussion (Section V CI explores 
the interactions between this feature and the qualitative 
measures of performance. 

In contrast with ‘Type’, the physics context of the 
problems (or lack thereof) was the least significant of the 
predictors for correctness {p = 0.86) and, as mentioned 


in Section III C the results for response time are likely 
inflated due to the way response time was measured (in¬ 
cluding the time to read the problem). 

The results for ‘Context’ are in contrast to Van Deven¬ 
ter [^, who found that on cross product direction ques¬ 
tions, the percentage of correct answers was significantly 
higher in a physics context (torque) than in a non-physics 
context. However, |Van Deventer| also found that the over¬ 
all correctness on cross product direction questions was 


3. Orientation 


As discussed in Section H C Scaife and Heckler [i] did 
not see any measurable effects on RHR use due to vec¬ 
tor orientation, yet research in spatial cognition (e.g., 
Klatzky and Wu 0 ) strongly suggests that the orienta¬ 
tion of the vectors should have an impact on performance. 
However, the spatial cognition literature also recognizes 
that orientation is not a single construct. By considering 
the physical discomfort, the plane of the vectors and the 
angle of the vectors as separate features, we see results 
that are consistent with the spatial cognition literature. 

The impact of both physical awkwardness (‘Comfort’) 
and the plane of the given vectors (‘Plane’) was sig¬ 
nificant at the p < 0.01 level for both correctness and 
response time, indicating that these features separately 
contribute to the difficulty of using a RHR. However, it 
is important to note that this study used the most physi¬ 
cally easy and the most physically awkward orientations. 
Therefore, it is possible that the effect from physical dis¬ 
comfort is inflated. Also, as will be discussed in the qual¬ 
itative analysis (Section |V C[ ), difficulty with vectors that 
are not in the xy-plane may also be due to issues other 
than mental rotation. 

The angle between the vectors (‘Angle’) also had a 
significant impact on correctness {p = 0.041), but not 
on response time {p = 0.14). The lack of a significance 
on response time is likely due to the fact that the angle 
was varied only for non-physics working-forward prob¬ 
lems in the xy-plane. Based on correctness and response 
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time, this subset of problems is one of the of the easiest 
(the overall level of correctness for this subset was above 
90%). Since there is a significant difference in correctness 
for even these easy problems, it is possible that on more 
difficult problems the angle would have an even stronger 
effect. Additionally, angle plays a key role in geometric 
methods for finding the magnitude of a cross product, 
which is not discussed here. Thus, the angle of the vec¬ 
tors should be considered in any future study examining 
student performance on cross product problems. 


Jf. Physics features 

The results from the physics features (‘Charge’ and 
‘Field’) demonstrate that within a physics context, the 
physical features of the situation do add to the chal¬ 
lenge of appropriately using a RHR. Both the sign of 
the charge and the representation of the field had a sig¬ 
nificant impact on correctness (p < 0.05), although not 
on response time. This difference in significance could 
indicate a speed-accuracy tradeoff. For example, if par¬ 
ticipants do not acknowledge the need to treat a negative 
charge differently than a positive charge, they will answer 
questions with negative charges as quickly as those with 
positive charges, but will answer them incorrectly. 


D. Summary of quantitative analyses 


The primary goal of the quantitative analysis was to 
provide a first-order look at the relationship between the 
features identified in the literature and the use of RHRs 
on cross product direction questions, as measured by cor¬ 
rectness and response time. The results of these analy¬ 
ses show that each feature does impact performance in 
a direction consistent with the hypotheses (Table |T| and 
while it is difficult to strictly interpret significance, some 
features have a greater effect than others. 

The type of reasoning required (‘Type’), all three as¬ 
pects of orientation (‘Comfort’, ‘Plane’, and ‘Angle’), and 
certain physics features (‘Charge’ and ‘Field’), all have 
a significant impact on student performance. Yet, it ap¬ 
pears that features involving parallel transport (‘Tails’ 
and ‘Separation’) may not have the same kind of impact 
on cross product problems that they do on vector addi¬ 
tion and subtraction problems. Finally, understanding 
the role of ‘Context’ will require more study to differ¬ 
entiate whether the differences between results here and 
previous research are due to an increased familiarity with 
cross products or with the specifics of the physics context. 


V. QUALITATIVE ANALYSES 


A. Overview of qualitative analyses 


As discussed in Section III C correctness and response 
time are fairly coarse measures of performance. Addi¬ 
tional insight can be gained by a more qualitative look 
at student use of RHRs. An iterative coding process 
was used to identify the methods that participants used 
to solve the problems and the types of errors that they 
made while doing so. 

An initial set of codes were derived using content logs 
of interviews, as well as a pilot study conducted the pre¬ 
vious semester. These codes were anchored in published 
research, but were then revised by a constant comparison 
with the data to determine if the codes fit the data. Once 
this revision process reached a point where it appeared 
that the codes were an accurate reflection of the data, a 
second independent coder was brought in to determine 
the reliability of the code definitions. This process led to 
further revisions to clarify definitions and further rounds 
of reliability testing. Each round of inter-rater reliability 
used 3-8% of the more than 2400 questions. To com¬ 
pensate for the smaller data set and to provide for more 
robust agreement, the problems used for interrater relia¬ 
bility were chosen to reflect the most difficult questions 
and the widest variety of codes. Since neither methods 
nor errors are mutually exclusive, each problem was given 
a code for every method and for each family of errors. 
Cohen’s kappa was calculated separately for each code 
family (ranging from 0.7-1.0). Kustusch [5] provides a 
more detailed description of the development and valida¬ 
tion of these coding schemes. 

Once all problems were coded for both methods and 
errors, the relationships between these methods and er¬ 
rors were explored, as well as how these methods and er¬ 
rors interacted with the various problem features of the 
tasks. Given the number of codes and problems, the re¬ 
sults of the quantitative analysis were used to focus this 
exploration on those problem features that had the most 
impact on correctness and response time. The following 
sections will present the most interesting and relevant of 
these codes and interactions, but a more complete anal¬ 
ysis can be found in Kustusch [B]. 


B. Results of qualitative analyses 

The method codes and sub-codes are graphically de¬ 
picted in Fig. Those in green will be discussed more 
fully in the text and brief definitions of these are provided 
in Table |V] Descriptions of all methods can be found in 
the appendix (Table [Vn| ). 

There were indications of two distinct families of meth¬ 
ods: primary and supplemental. Primary methods can, 
on their own, yield a singular, non-zero answer and the 
most common were the various RHRs. There was the 
occasional use of matrices, as well as vector addition and 
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Method Codes 


[ Primary Methods 


Addition 
or Matrix 


1 


Right-hand Rule 


IC 


1 


Type 




Right 


Supplemental Methods 


3 fingers 


— ^ Guess and Check J 
— ^ Rotation ] 

— I Diagram j 
— I Multiplication j 
— [ Parallel TransportJ 
— I Orthogonality j 
—I Physics Knowledge j 


FIG. 8. Graphical depiction of method codes and sub-codes. 
Those in green are discussed in the text and in Table [V] 


subtraction methods, but neither of these will be ad¬ 
dressed in this paper. Supplemental methods are those 
that when used on their own cannot yield a singular, 
non-zero answer. They are the tools, techniques, and 
strategies that the participants usually used in conjunc¬ 
tion with a primary method. For example, using the 
orthogonality of the cross product to restrict the solu¬ 
tion to one axis and then using a RHR to determine the 
direction along that axis. 


Greenslade [T] points out that there are numerous 
physical mnemonics for the cross product that can vary 
widely. For the purpose of this study, all such mnemon¬ 
ics have been grouped as Right-hand Rules (or RHR) and 
identified with three sub-codes: the Hand that was used 
(‘Left’ or ‘Right’); the Frame of reference with which 
the hand was aligned (‘Table’ or ‘Other’), and the gen¬ 
eral Type. For example, the RHR described by Klatzky 
and Wu |3] is called the ‘Standard’ right-hand rule. The 
two others that will be discussed in this paper are the 
‘Current’ rule, where the thumb points in the direction 
of the current and the fingers curl to the observation lo¬ 
cation, and the ‘3-Fingers’ rule, where the index finger 


points along H, the midd le finger along B and the thumb 
along C (see Table VII for descriptions of all types). If 


any of these sub-codes changed during the problem, it 
was considered a different use of a RHR. Thus, RHR was 
the only method code used more than once per problem. 

The two supplemental methods that will be discussed 
in the following section are Guess-and-Check and Rota¬ 
tion. As the name indicates. Guess-and-Check involved 
making a guess or assumption about the answer and us¬ 
ing another method (usually a RHR) to check if the guess 
was consistent with the given information. Rotation in¬ 
volved rotating the given frame of reference, usually by 
physically rotating the paper, but also sometimes by re¬ 
drawing the given diagram rotated along some axis. 


TABLE V. Brief definitions for methods discussed in the text. 
Descriptions of all methods can be found in Table [VTIl 


Code Name 

Code Definition 

RHR:Hand 

Specifies whether participant uses the 
‘Right’ or ‘Left’ hand 

RHR: Frame 

Specifies whether the right-hand rule is per¬ 
formed in the plane of the table (‘Table’), 
where -l-a; is to the right, -\-y is away from the 
participant, and -\-z vertically upward, or in 
a different (‘Other’) frame, e.g., -{-z is toward 
participant instead of vertically upward. 

RHR:Type: 

‘Standard’ 

Fingers point in the direction of the first vec¬ 
tor (A), curl in the direction of the second 
vector (B) and the thumb points in the di¬ 
rection ot A X B. 

RHR:Type: 

‘Current’ 

Thumb points in the direction of conven¬ 
tional current and the fingers curl such that 
at a given observation location, the fingers 
point in the direction of the magnetic field 
at that location. 

RHR:Type: 

‘S-Finger’ 

The index finger points in the direction of 
the first vector (A), the middle finger points 
in the direction of the second vector (B) and 
the thumb points in the direction of A x B. 

Guess-and- 

Check 

Participant explicitly “guesses” an answer or 
makes an assumption (e.g., a particle has 
positive charge) and uses another method to 
determine whether the assumption is consis¬ 
tent with the given information. 

Rotation 

Given frame of reference is rotated by more 
than « 15° using the paper or a diagram. 


The full set of error types are shown in Fig. The 
codes in green are discussed in this paper and described 
in Table |Vl| Brief descr iption s for all codes can be found 
in the appendix (Table [VHI ). Errors were grouped into 
three larger categories: errors associated with a primary 
method (only RHR errors will be discussed here), physics 
errors, and all other types of errors. Since the different 
types of errors are not mutually exclusive, every problem 
received an code for each type of error and each unique 
use of a RHR received a corresponding error code. 


When an error type contributed to the final response, 
it was coded as ‘Error’. However, there were times when 
an error was made, but it did not contribute to the final 
response. This typically occurred because the partici¬ 
pant recognized the error and corrected it or abandoned 
the approach that led to that error — these situations 
were coded as ‘Corrected’. A code of ‘No Error’ indi¬ 
cated that there was no evidence for that type of error 
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FIG. 9. Graphical depiction of error codes and sub-codes. Godes in green are discussed in the text and described in Table m 
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TABLE VI. Brief definitions of the error types discussed in 
the text. Descriptions of all error types can be found in Ta- 
ble |VIIl| The definitions here correspond to ‘Error’ sub-codes, 
where sub-codes of ‘Corrected’ or ‘No Error’ are also possi¬ 
bilities. For Charge and Electric Field, explicit and implicit 
errors were categorized separately. 


Code Name 

Code Definition 

RHR: 

Inappropriate 

Participant attempts to make use of a RHR 
that is inappropriate for the given situation. 

RHR: 

Direction 

Participant inappropriately reverses the direc¬ 
tion of one of the given vectors. 

RHR: 

Order 

Participant performs the RHR in such a way 
that the order of the vectors is reversed and 
they are performing B x A instead of A x B. 

Observation 

Location 

Participant does not appropriately account for 
the observation location. 

Charge 

Participant does not appropriately account for 
the sign of the charge. 

Electric Field 

Participant explicitly reasons from or provides 
an answer consistent with, an invalid or inap¬ 
propriate analogy to electric field. 

Misinterpret 

Participant misinterprets the diagram or mis¬ 
reads the problems statement in a way that 
contributes to the final response. 


for that problem. Thus, a problem that received ‘No Er¬ 
ror’ codes for all types was answered correctly, although 
the converse was not necessarily true due to corrected 
errors and multiple errors that canceled each other out. 

For some types of errors, there were examples with im¬ 
plicit evidence {e.g., not accounting for the sign of a neg¬ 
ative charge) and examples with explicit evidence {e.g., 
explicitly double counting the negative sign for problems 
involving an electron). For Charge and Electrie Field 
errors, implicit and explicit error were categorized sepa¬ 
rately, but for Observation Location errors, explicit and 
implicit errors were grouped together. 

Based on the coding described here, patterns emerged 
that were associated with certain errors, methods and/or 
problem features. The most relevant of these patterns 
will be presented and discussed in the following section. 


C. Discussion of qualitative analyses 

This section discusses five aspects of the qualitative 
coding and analyses that build on the quantitative anal¬ 
yses in Section [IV| by providing: 

1. a more nuanced perspective on student difficulties 
with the commutativity of the cross product. 


2. an alternate explanation for student difficulties 
with non-cry plane orientations, 

3. a look at a supplemental method {Rotation) that 
appears to mitigate difficulties with orientation, 

4. the verification of several previously identified dif¬ 
ficulties with physics content, and 

5. the identification of physics difficulties that have 
not previously been discussed in the literature. 

These outcomes demonstrate the complexity of how con¬ 
textual and representational features impact student use 
of right-hand rules. 


1. Adding nuance to the issue of commutativity 


The largest number of uncorrected errors associated 
with a RHR were Order errors. This is a type of error 
where the participant either implicitly or explicitly re¬ 
verses the order of the vectors (i.e., C = B x A instead of 
C = Ax B). The existence and prevalence of this kind of 
error gives credence to the claim by Scaife and Heckler [3] 
that one of the causes of “sign errors” is that students do 
not recognize the non-commutativity of the cross prod¬ 
uct. However, in their study, sign errors committed by 
students who treated the cross product as commutative 
appeared to non-systematic. By looking at the Order er¬ 
rors in the current study more closely, we can illuminate 
the complexity of this issue of commutativity. 

As mentioned in Section IHI Al the interviewer asked 
each participant at the end of the interview how B x A 
compared to Ax B. Many participants stated that these 
two quantities would have the same magnitude, but point 
in opposite directions. Yet even these students commit¬ 
ted Order errors. So, while they may acknowledge the 
non-commutativity of the cross product when asked di¬ 
rectly, they still sometimes operationally treat the cross 
product as commutative. Examining the interaction be¬ 
tween Order errors and the various problem features re¬ 
veals one possible reason for this disconnect. 

The quantitative results in Section |IV| identified the 
type of reasoning as a feature with a notable impact 
on correctness and response time. Therefore, it should 
not be surprising that more than 70% of the Order er¬ 
rors were committed on Backward problems (see Fig. 101. 
What was slightly more surprising was that more than 
half of all Order errors were on problems where the given 
vectors were B and C and the partic ipan t was asked to 
find A, such as the one shown in Fig. [idj Even students 
who performed well on all problems (> 90%) and who 
explicitly identified the non-commutativity of the cross 
product committed this type of error {Order) on this 
kind of working backward problem (finding A). 

Barry was one such participant who generally per¬ 
formed well. On the problem in Fig. [m Barry made 
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FIG. 10. The number of RHR: Order errors by the type of 
reasoning required. Backward problems are separated by the 
type of answer required, i.e., A means that the first vector in 
the cross product was required, B means the second vector 
was required, and q means that all vectors were provided and 
the sign of the charge was required. 


an Order error, but then corrected himself. When asked 
what made the problem difficult, he stated, 

“Gosh, it’s hard to start without the initial 
direction, because what I see on the page is 
two points and so I’m thinking this [points 
fingers to the left] automatically, but really, 
that’s the second step in finding a direction.” 

This ‘automatic’ response that Barry identifies is con¬ 
sistent with the idea that problems which require work¬ 
ing backward have a higher cognitive load than those 
that require working forward. It suggests that the Or¬ 
der errors committed by those who do understand the 
non-commutativity of the cross product may be at least 
partially the result of not appropriately compensating for 
the backward nature of the problem. 

It is interesting that when Guess-and-Check was used 
as a supplemental method on working backward prob¬ 
lems, the percentage of Order errors was only 8% com¬ 
pared to 15% when Guess-and-Check was not used. Ad¬ 
ditionally, Guess-and-Check was used almost exclusively 
on Backward problems, suggesting that participants var¬ 
ied their methods in part to alleviate the challenges of 
this particular problem feature. 

We can also look at individuals, such as Andrea, who 
explicitly stated that BxA = AxB to better explore 
the non-systematic nature of these errors. Examining the 
features of the problems where Andrea made Order er¬ 
rors and those where she did not, it became clear that 


Problem: The magnetic field due to the negative charge 
is in the —a direction (into the page) at the obser¬ 
vation location shown. What is a possible direction 
for the velocity of the charge? 


B 




FIG. 11. A magnetic field problem that requires reasoning 
backward to find the first vector in the cross product. 


while her belief in the commutativity of the cross product 
contributed to these errors, the errors were by no means 
arbitrary. Every problem where she committed an Order 
error was one where the orientation was physically awk¬ 
ward. Andrea consistently used the ‘3-Finger’ RHR and 
on physically awkward problems, she would often reverse 
which finger represented which vector, essentially revers¬ 
ing the order of the cross product. For her, the belief that 
order did not matter did not create an arbitrary error, 
but one that was a way of dealing with the physical awk¬ 
wardness of some of the orientations. This interaction 
between certain problem features and Order errors for 
those that do not understand non-commutativity could 
partially account for the apparent non-systematic sign 
errors observed by Scaife and Heckler [4j. 

The patterns discussed here demonstrate that while 
the connection between Order errors and commutativ¬ 
ity is real, this interaction is often mediated by prob¬ 
lem features, such as the type of reasoning required and 
the physical awkwardness of using a RHR, and by other 
methods, such as Guess-and-Check. 


8. Alternative explanation for non-xy orientations 


As mentioned in Section IV the results of the quan¬ 
titative analysis show that when the orientation of the 
vectors is treated as more than a single construct, each 
aspect of orientation does impact student performance as 
we would expect based on the spatial cognition perspec¬ 
tive. Based on literature on mental rotations, we expect 
that participants would perform better on problems in 
the xy plane (where the axis of rotation is vertical) than 
on those in a non-xy plane. While this trend is seen in the 
quantitative analysis, the qualitative analysis revealed an 
additional issue related to the plane of the vectors that 
is not directly related to mental rotations. 

There were two types of errors that were more common 
on non — xy plane problems {i.e., one of the given vectors 
was into or out of the page in the ±z direction) than on 
xy plane problems. One of these errors was the RHR: 
Direction error, where the participant implicitly reversed 
the direction of one of the vectors while attempting a 
RHR. Due to the implicit nature of this error, it may 
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Problem: What is the direction of A x B? 


mitigate difficulties associated with orientation. 



B 

FIG. 12. An example problem where the given vectors are in 
the non-xy plane. 


be the participants are unaware that they are actually 
doing it. The other error was a Misinterpret error, where 
the participant explicitly states a wrong direction, but 
performs a RHR appropriately for the stated direction. 

Consider Fig. [T^ to understand the difference between 
these two error types. If one were to use the ‘Standard’ 
RHR on this problem, the fingers should point into the 
page and curl down. If the participant said that A was 
into the page, but pointed the fingers out of the page 
(reversing the direction of A), this would be considered 
a Direction error. If, however, the participant explicitly 
stated that A was out of the page and pointed the fingers 
out of the page, it would be coded as a Misinterpret error. 

The existence of Misinterpret errors and their preva¬ 
lence on problems where one of the vectors is along the 
z axis suggests an additional reason for poorer perfor¬ 
mance on non-cry plane problems. While some of the er¬ 
rors on these problems may be related to difficulties with 
mental rotations, there is likely also an issue related to 
understanding the and 0 symbols for into and out of 
the page, which are conventionally used to show a three- 
dimensional situation in a two-dimensional plane. With 
the current data, it is not possible to disentangle which 
is the more prominent source of difficulty, but it raises 
an issue that should be addressed in the future research 
using these symbols. 


3. Mitigating dijficulties with orientation by rotation 


As mentioned above (Section VCl), when students 
used the Guess-and-Check method on Backward prob¬ 
lems, the percentage of Order errors was smaller than 
when they didn’t use Guess-and-Check. There was a sim¬ 
ilar effect from using Rotation for two of the orientation 
features: ‘Comfort’ and ‘Plane’. As shown in Fig. [1^ 
when Rotation was used, the fraction of RHR errors is 
smaller overall compared to when Rotation was not used. 
In addition, the difference between Easy and Awkward 
problems is reduced (Fig. [^a), as is the difference be¬ 
tween xy plane and non-xy plane problems (Fig. &). 

Some instructors already encourage their students to 
physically rotate the paper if necessary to allow them to 
more easily perform a RHR. The findings presented here 
support the claim that this approach does indeed help to 
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FIG. 13. Interaction of rotation and orientation: (a) fraction 
of RHR errors on physically easy and awkward problems with 
and without rotation; (b) fraction of RHR errors on problems 
in the xy and non-xy planes with and without rotation. 


4- Previously identified difficulties with physics content 

Not surprisingly, there were several types of errors that 
were more connected to the physics than to the mathe¬ 
matics of these problems. Some of these errors have been 
previously discussed in the literature on student under¬ 
standing of electromagnetism and some were associated 
with particular physics features of the problems. 

Consistent with the quantitative analyses and with the 
findings of Scaife and Heckler mn], there were two kinds 
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of errors that were more associated with problems with 
Magnet (or magnetic pole) representation for the mag¬ 
netic field than with the Vector representation. The first 
type of error was when the participant treated the di¬ 
rection of the magnetic held as pointing from South to 
North instead of from North to South (either a Direction 
or Misinterpret error). 

The second error involved an inappropriate analogy 
to electric helds and forces {Electric Field), although the 
analogy was rarely stated explicitly. In the Magnet repre¬ 
sentation, several students stated that the magnetic force 
on a moving charge would be parallel to the held. For 
example, Ana answered that 

“The force would be down... negative y, be¬ 
cause it’s pulled towards... the south and 
pushed away from north.” 

This kind of implicit analogy with electric held was 
also associated with problems that asked for the magnetic 
held at the center of a loop of current-carrying wire. On 
these problems, eight of the participants responded at 
least once like Danny did, 

“... since it’s in the center, they’re all going 
to cancel out, so it just has to be... zero mag¬ 
nitude.” 

The fact that this type of error only appeared on prob¬ 
lems with certain features suggests that there is some¬ 
thing about the features that cued inappropriate electric 
held resources for these students. This interpretation is 
consistent with the bi-directional interference effects be¬ 
tween electric and magnetic force noted by Scaife and 
Heckler |12) . 

For the 2nd group of students in this study who were 
interviewed later in the term, there was also some ev¬ 
idence of another interference effect, between magnetic 
held and magnetic force. For example, when Danny was 
trying to hnd the magnetic held produced by a mov¬ 
ing charged particle, he tended to inappropriately try to 
apply a RHR designed to hnd the magnetic force on a 
moving charged particle. While not conclusive, this type 
of confusion implies that some students may not yet have 
diherentiated the two different cross products (and asso¬ 
ciated right-hand rules) for these situations. 

The issues discussed here are consistent with previous 
literature on student understanding of electromagnetism 
and provide additional evidence of the prevalence of some 
of these student difficulties. 


5. Previously unidentified difficulties with physics content 

There were two physics errors identihed in this data set 
have not been previously discussed in the literature, but 
are consistent with anecdotal evidence from instructors. 

The hrst error, consistent with the quantitative analy¬ 
sis, was that participants would not always appropriately 


account for a negative charge. Sometimes this meant sim¬ 
ply ignoring the sign of the charge, whereas other times, 
participants might “double-count” the negative charge. 
Of those that did not have Charge errors, two (Fred and 
Danny) explicitly used a left-hand rule when there was a 
negative charge. Some students also expressed confusion 
about conventional versus electron current, which may 
be unique to Matter and Interactions curriculum |44j . 

The second error involved not appropriately account¬ 
ing for the observation location on magnetic field ques¬ 
tions. What makes this error even more interesting is 
that it appeared to be particularly associated with the 
‘Current’ RHR, where one points the thumb in the di¬ 
rection of the current and curls the hngers to the ob¬ 
servation location. There were two participants (Ericka 
and Sebastien) who typically used this RHR and consis¬ 
tently did not curl their hngers to the appropriate loca¬ 
tion. Given that this particular RHR does not explicitly 
require identifying the observation location, it seems rea¬ 
sonable that those using this type of RHR might be more 
prone to this kind of error. 


D. Summary of qualitative analyses 

Expanding our definition of performance to include the 
methods that participants used and the types of errors 
that they made provides an additional window into how 
the contextual and representational features impact per¬ 
formance. 

First, even students who state that the order of vectors 
in a cross product is important may switch this order. 
This happens particularly on problems requiring back¬ 
ward reasoning where you are looking for the first vector 
in the cross product. Using a Guess-and-Check supple¬ 
mental method for this kind of a problem may help to 
mitigate the difficulty. In addition, for students that do 
not recognize the non-commutativity of the cross prod¬ 
uct, errors may be mediated by other problem features, 
such as the physical awkwardness of the orientation. 

Second, the quantitative analyses shows that students 
perform more poorly on problems with non-xy plane ori¬ 
entations than on problems with xy plane orientations. 
However, the qualitative analysis reveals that this trend 
may not be due solely to difficulties with mental rota¬ 
tions. Instead, the trend may be partly due to not cor¬ 
rectly interpreting the symbols for vectors aligned with 
the 2 :-axis (0 and Q). 

Third, there is evidence that physically rotating the pa¬ 
per may help to mitigate difficulties associated with spe¬ 
cific problem features associated with orientation, such 
as the physical awkwardness or an orientation that is not 
aligned with the vertical. 

Fourth, the categorization of error types provides addi¬ 
tional support for previous research on student difficulties 
with the Magnet (or magnetic poles) representation, as 
well as for interference effects between electric and mag¬ 
netic fields and between magnetic field and force. 
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Fifth, there are two errors associated with the physics 
contexts that have not previously been discussed in the 
literature, but are consistent with instructor experience: 
students struggle to appropriately account for negative 
charges (and electron currents) and the observation loca¬ 
tion when using RHRs. In particular, errors in account¬ 
ing for the observation location seem to be associated 
with a particular right-hand rule for some students. 


VI. CONCLUSIONS AND IMPLICATIONS 


2. Parallel Transport: Tails and Separation 

In the light of the literature on vector addition and 
subtraction, it was somewhat surprising to see that the 
features associated with parallel transport appeared to 
have little effect on performance. It may be that by this 
point in the physics sequence, parallel transport is no 
longer such a stumbling block to students. Alternatively, 
parallel transport may not be as necessary for using a 
right-hand rule as it is for graphical vector addition and 
subtraction. 


This study sought to answer the question: 


3. Orientation: Comfort, Plane, and Angle 


How do contextual and representational fea¬ 
tures impact student performance on prob¬ 
lems requiring the use of right-hand rules? 


Previous research was used to identify ten different prob¬ 
lem features that were likely to impact performance 
(Fig. §. Then four different measures of performance 
and both quantitative and qualitative analyses were em¬ 
ployed to address how those ten features impacted stu¬ 
dents use of RHRs. The quantitative analysis was pri¬ 
marily used to identify problem features that have the 
greatest impact on performance, as measured by response 
time and correctness. These results provided a focus for 
the qualitative analysis, which in turn revealed some of 
the nuances associated with the representational depen¬ 
dence of performance. Here we briefly summarize the 
findings from both analyses (Section [VIA ) and discuss 
the implications for instruction (Section VIBI 


A. Review of findings 

1. Kind of question: Context and Type 

Of all of the many features explored in this study, 
the type of reasoning required (working forward versus 
working backward) had the strongest impact on multi¬ 
ple aspects of performance: response time, correctness, 
and number of RHR errors made. All of these results 
show that working backward is cognitively more challeng¬ 
ing than working forward, and in particular, finding the 
first vector of a cross product is the most difficult back¬ 
ward problem. The additional cognitive load of this type 
of problem may account for students treating the cross 
product as commutative even when they acknowledge the 
non-commutativity in other contexts. However, there are 
supplemental methods, such as Guess-and-Check which 
may help to reduce this cognitive difficulty. 

The impact of context is less clear and requires further 
study, but there is support for previous research suggest¬ 
ing interference effects between different physics contexts 
(z.e., electric vs. magnetic, field vs. force, etc.). 


By treating the orientation of the vectors as three sep¬ 
arate constructs instead of one, this study has resolved 
an apparent difference between the spatial cognition lit¬ 
erature and Scaife and Heckler [i] on the impact of ori¬ 
entation. The quantitative analysis here showed that the 
degree of physical awkwardness, the plane of the vectors, 
and the angle of the vectors each have an impact on per¬ 
formance consistent with the spatial cognition literature. 

However, the qualitative analysis revealed that spatial 
cognition may not the only factor contributing to diffi¬ 
culties with orientations. The existence of Misinterpret 
errors and their prevalence on problems with non-xy ori¬ 
entations indicates that some of the difficulty with the 
plane of the vectors may stem from the symbols used 
to represent 3-dimensional vectors on a 2-dimensional 
page. In addition, one student, Andrea, was more likely 
to switch the order of the vectors on problems with phys¬ 
ically awkward orientations, highlighting her belief in the 
commutativity of the cross product. This suggests that 
certain orientations may elicit errors that might other¬ 
wise not be apparent. 

As with backward reasoning problems, there is evi¬ 
dence of a supplemental method reducing errors associ¬ 
ated with orientation. Physically rotating the paper re¬ 
duced the overall fraction of errors, as well as the gap be¬ 
tween easy and challenging orientations. This trend was 
seen both with the different levels of physical awkward¬ 
ness and with the different planes of the given vectors. 


f. Physics features: Charge, Shape, and Field 

This study identified several significant physics errors 
and related these errors to certain physics features. Each 
of these findings is consistent with previous literature and 
in some cases, provides support for conventional wisdom 
that has been anecdotal until now. 

This study confirmed the impact of the representa¬ 
tion of magnetic field observed by Scaife and Heckler 
min], where participants perform more poorly on ques¬ 
tions with a bar magnet (or magnetic pole) representa¬ 
tion than on questions with a vector representation. The 
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physics errors associated with this feature are also consis¬ 
tent with their findings. The first involved reversing the 
direction of the magnetic field and the second involved 
believing the force to be parallel to the field, likely using 
an inappropriate analogy to electric field. This confusion 
between electric and magnetic concepts, which has been 
explored in several previous studies nnumis]), was also 
seen on magnetic field problems where participants would 
state that the field “points away from the positive and 
toward the negative” or that the field at the center of a 
loop was zero. 

The most significant physics errors in this data was 
not appropriately accounting for the sign of the charge. 
While most of these errors resulted from participants ig¬ 
noring or neglecting the issue, some participants explic¬ 
itly did not understand how the sign of the charge would 
impact the cross product and several students expressed 
confusion about conventional versus electron current. 

Finally, a previously un-explored issue in the correct 
application of RHRs is the effect of the observation lo¬ 
cation on the direction of the magnetic field. While not 
associated with a particular problem feature, observa¬ 
tion location errors were one of the few instances where 
a specific type of right-hand rule was associated with a 
particular error. The majority of observation location 
errors were made when using the ‘Current’ right-hand 
rule and for at least two of the participants (Ericka and 
Sebastien), this was a systemic issue. 


B. Implications for instruction 


For instructors who want to help students use RHRs 
effectively, this study has two primary implications: 

1. Context and representation matter and it is impor¬ 
tant to attend to the features of a problem when 
thinking about instruction and assessment, and 

2. There are other resources available that can be 
leveraged to alleviate some of the difficulties asso¬ 
ciated with certain problem features. 


The quantitative analysis showed several contextual 
and representational features that do, indeed, impact stu¬ 
dent performance and the qualitative analysis provided 
a more in-depth look at how they do so. For example, 
the problem in Fig. 11 has several features that make it 
significantly more difficult than the problem in Fig. [T^ 
it is a physics problem that requires working backward 
and involves a negative charge. We want students to be 
able to perform appropriately on problems with different 
features, but we also need to be aware of the relative diffi¬ 
culty of the problems we assign and why we are assigning 
them. Providing a variety of orientations or scaffolding 
from more cognitively easy features to more difficult ones 
may help students to practice the skills that they need 
to be able to use RHRs appropriately in multiple con¬ 
texts and with different representations. In addition, we 


should attend to possible interference effects from other 
concepts that students are learning, e.g., dot products vs. 
cross products, electric concepts vs. magnetic concepts, 
field vs. forces, etc. 

In addition to attending to the contextual and repre¬ 
sentational features of the problems we use and assign, 
we can and should leverage additional resources to pro¬ 
vide students with additional tools that can help to ease 
some of the difficulties associated with some of these fea¬ 
tures. We can start with the tools that the participants 
in this study were using: Guess-and-Check as a way to 
reduce errors on Backward problems and Rotation as way 
to help with orientation issues. 

The role of Rotation in reducing errors associated with 
orientation suggests that manipulatives, such as those de¬ 
signed by Van Domelen and Nguyen and Meltzer [T^ 
could be effective in addressing orientation issues as well. 
Another option would be to leverage the spatial nature 
of the classroom through embodied learning activities. 
For example, one might have each student represents a 
point in space and use their bodies to show the direction 
of the field (or force) at their location due to a charge or 
current at some other location m- This would not only 
emphasize that the field is different at different locations, 
but would also give the instructor formative assessment 
about each student’s ability to use a RHR appropriately. 
One productive area for future inquiry would be to assess 
the effectiveness of such tools and activities. 

Also, given the rise of online resources, it is also im¬ 
portant to explore how the impact of orientation would 
change for tasks on a computer, such with an on¬ 
line homework system (e.g., WebAssign or Mastering 
Physics) or with a computer simulation. The computer 
reference frame is more consistent with instructors’ use 
of whiteboards, but exams are typically given on paper. 
The inability to rotate a 2-dimensional image on a com¬ 
puter screen as one can rotate a piece of paper could 
present challenges, but the use of rotatable computer 
simulations could provide additional tools for students 
to explore the 3-dimensional nature of cross products. 

Finally, the difficulty that some participants had in ap¬ 
propriately accounting for the observation location is an 
issue that has not previously been discussed as such in the 
literature and has potentially far reaching implications. 
For physics majors, much of upper-division electromag¬ 
netism involves exploring and assessing patterns of fields 
in space where understanding the role of the separation 
vector (r — r') is vital. Thus, it is important to explore 
to what extent the issue identified here is due to an inap¬ 
propriate application of a RHR and to what extent this 
is a more foundational conceptual difficulty with the role 
of the observation location. 

This study has demonstrated that when it comes to us¬ 
ing a right-hand rule appropriately, the contextual and 
representational features make a difference. If we want to 
help students to use right-hand rules effectively, we may 
need to leverage other resources to help to ease the cog¬ 
nitive difficulties associated with these representations. 
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TABLE VII. Definitions for all methods (see Kustusch [6] for more detailed code definitions and examples) 


Code Name Code Definition 


RIGHT-HAND RULE (RHR) Participant uses a physical technique to find the direction of the cross product 


GUESS & CHECK 

Sub-code Definition 

HAND Specifies whether participant uses RIGHT or LEET hand 

FRAME Specifies whether the RHR is performed in the plane of the table (TABLE), where 
-\-x is to the right, -\-y is away from the participant, and +z vertically upward, or 
in a different (OTHER) frame, e.g., +z is toward participant instead of vertically 
upward. 

TYPE Specifies the type of right-hand rule. 

STANDARD: Fingers point in the direction of the first vector (A), curl in the 
direction of the second vector (B) and the thumb points in the direction of A x B. 

CURRENT: Thumb points in the direction of the conventional current and the 
fingers curl in the direction of the magnetic field, such that at a given observation 
location, the fingers point in the direction of the magnetic field at that location. 

LOOP: Fingers curl around a loop of wire and thumb gives the direction of the 
magnetic field at the center of the loop. 

3 FINGERS: The index finger points in the direction of the first vector (A), the 
middle finger points in the direction of the second vector (B) and the thumb 
points in the direction of A x B. 

KNUCKLES: Knuckles point in the direction of the first vector (A), fingers in the 
direction of the second vector (B) and thumb points in the direction of A x B. 

PALM: Thumb points in the direction of the first vector (A), fingers in the direc¬ 
tion of the second vector (B) and palm points in the direction of A x B. 

Participant explicitly “guesses” an answer or makes an assumption (e.g., a particle has positive 
charge) and uses another method to determine whether the assumption is consistent with the 
given information. 

ROTATION 

The given frame of reference is rotated by more than « 15° using either the paper or a diagram. 

DIAGRAM 

Participant draws a diagram. 

MULTIPLICATION 

Participant uses either commutative or non-commutative multiplication rules outside of an 
explicit calculation (e.g., two negatives make a positive). 

PARALLEL TRANSPORT 

Participant explicitly mentions (or gestures or draws) moving the vectors. 

ORTHOGONALITY 

Participant restricts the solution to one axis before using a right-hand rule (e.g., since given 
vectors are along x and y axes, the cross product must be ±i). 

PHYSICS KNOWLEDGE 

Participant explicitly mentions physics not present in the problem statement or uses physics 
information from the problem statement to reason about the problem. 

ADDITION 

Participant uses methods consistent with vector addition or subtraction. 

MATRIX 

Participant makes use of matrices and determinants in finding the direction of the cross product. 

UNCLEAR 

Not enough evidence to determine what method(s) participant used to solve the problem. 
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TABLE VIII. Definitions for all errors types (see Kustusch for more detailed code definitions and examples). Unless 
otherwise specified, the definition corresponds to an ERROR sub-code, where sub-codes of CORRECTED or NO ERROR are 
also possibilities. For CHARGE and ELECTRIC FIELD, explicit and implicit errors were categorized separately. 


Code Name 

Code Dehnition 

ADDITION 

Participant uses vector addition or subtraction method (e.g., connects tip to tail or tip to tip). 

MATRIX 

Participant makes at least one error with the use of matrices that contributes to hnal response. 

RIGHT-HAND RULE 

Sub-code Definition 

NO ERROR Participant makes no errors with this RHR regardless of whether the 

final response includes an error based on another method. 

INAPPROPRIATE Participant attempts to make use of a right-hand rule that is inappropri¬ 
ate for the given situation. If any other error (e.g. Direction or Order) 
is made, this code takes precedence. 

DIRECTION Participant inappropriately reverses the direction of one of the given vec¬ 

tors by explicitly stating correct direction and using the wrong direction 
or by using the wrong direction without stating it. 

ORDER Participant does the right hand rule in such a way that the order of the 

vectors is reversed and they are performing B x A instead of A x B. 

RHR OTHER Participant uses a right-hand rule such that it is unclear what error(s) 

they are making or the error does not fall into one of the other right-hand 
rule categories or they make multiple errors. 

OBSERVATION 

LOCATION 

Participant does not appropriately account for the observation location, either implicitly or 
explicitly (categorized together). 

CHARGE 

Participant does not appropriately account for the sign of the charge, either implicitly or 
explicitly (categorized separately). 

ELECTRIC FIELD 

Participant explicitly reasons from, provides an answer consistent with, an invalid or inappro¬ 
priate analogy to electric field (implicit and explicit errors categorized separately). 

PHYSICS OTHER 

Participant explicitly invokes incorrect or inappropriate physics not covered in the other 
“Physics” categories or uses the same wrong physics on question(s) immediately following 
an explicit error without restating the physics. 

MISINTERPRET 

Participant misinterprets the diagram or misreads the problems statement in a way that con¬ 
tributes to the final response. 

MATCH 

Participant gives a verbal solution that does not match the gestural (or diagrammatic) response. 

OTHER 

There is at least one error that contributes to the final response which is not due to a primary 
method, physics, misinterpretation or a gestural mismatch. 


[49] A more detailed discussion of the control variables can AAPT, Minneapolis, MN (2014). 

be found in Kustusch [6]. [51] D. Hestenes, Am. J. Phys. 71, 104 (2003) 

[50[ M. B. Kustusch and S. Fischer, “Sequencing kinesthetic 
activities to explore fields and forces,” contributed talk. 








